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Important notice 
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The present document may be made available in more than one electronic version or in print. In any case of existing or 

perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF). 

In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive 

within ETSI Secretariat. 

Users of the present document should be aware that the document may be subject to revision or change of status. 

Information on the current status of this and other ETSI documents is available at 

http://portal.etsi.orq/tb/status/status.asp 

If you find errors in the present document, please send your comment to one of the following services: 

http://portal.etsi.orq/chaircor/ETSI support.asp 

Copyright Notification 

No part may be reproduced except as authorized by written permission. 
The copyright and the foregoing restriction extend to reproduction in all media. 
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Intellectual Property Rights 



IPRs essential or potentially essential to the present document may have been declared to ETSI. The information 
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found 
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in 
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web 
server ( http://webapp.etsi.org/IPR/home.asp ). 

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee 
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web 
server) which are, or may be, or may become, essential to the present document. 



Foreword 

This Technical Specification (TS) has been produced by ETSI 3rd Generation Partnership Project (3GPP). 

The present document may refer to technical specifications or reports using their 3GPP identities, UMTS identities or 
GSM identities. These should be interpreted as being references to the corresponding ETSI deliverables. 

The cross reference between GSM, UMTS, 3GPP and ETSI identities can be found under 
http://webapp.etsi.org/kev/quervform.asp . 
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Foreword 



rd , 



This Technical Specification (TS) has been produced by the 3 Generation Partnership Project (3GPP). 

The contents of the present document are subject to continuing work within the TSG and may change following formal 
TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an 
identifying change of release date and an increase in version number as follows: 

Version x.y.z 

where: 

X the first digit: 

1 presented to TSG for information; 

2 presented to TSG for approval; 

3 or greater indicates TSG approved document under change control. 

y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections, 
updates, etc. 

z the third digit is incremented when editorial only changes have been incorporated in the document. 
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Scope 



The present document specifies the digital test sequences and conformance criteria for the Enhanced aacPlus audio 
codec. 



Normative references 



The following documents contain provisions which, through reference in this text, constitute provisions of the present 
document. 

• References are either specific (identified by date of publication, edition number, version number, etc.) or 
non-specific. 

• For a specific reference, subsequent revisions do not apply. 

• For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including 
a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same 
Release as the present document. 

[1] 3GPP TS 26.401: "General audio codec audio processing functions; Enhanced aacPlus general 

audio codec; General description". 

[2] 3GPP TS 26.403: "General audio codec audio processing functions; Enhanced aacPlus general 

audio codec; Encoder specification; Advanced Audio Coding (AAC) part". 

[3] 3GPP TS 26.404: "General audio codec audio processing functions; Enhanced aacPlus general 

audio codec; Encoder specification; Spectral Band Replication (SBR) part". 

[4] 3GPP TS 26.405: "General audio codec audio processing functions; Enhanced aacPlus general 

audio codec; Encoder specification; Parametric stereo part". 

[5] 3GPP TS 26.410: "General audio codec audio processing functions; Enhanced aacPlus general 

audio codec; Floating-point ANSI-C code". 

[6] 3GPP TS 26.411: "General audio codec audio processing functions; Enhanced aacPlus general 

audio codec; Fixed-point ANSI-C code". 

[7] ISO/IEC 14496-4:2004: "Information technology - Coding of audio-visual objects - 

Part 4: Conformance testing". 

[8] ISO/IEC 14496-4:2004/FDAM 8:2004: "Information technology - Coding of audio-visual objects - 

Part 4: Conformance testing - Amendment 8". 

[9] ITU-R Recommendation BS. 1387-1: "Method for objective measurements of PErceived Audio 

Quality (PEAQ)". 
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3 Definitions and abbreviations 

3.1 Definitions 

For the purposes of the present document, the terms and definitions given in 3GPP TS 26.401 [1], 3GPP TS 26.403 [2], 
3GPP TS 26.404 [3], 3GPP TS 26.405 [4], 3GPP TS 26.410 [5]and 3GPP TS 26.41 1 [6] apply. 

3.2 Abbreviations 

For the purposes of the present document, the following abbreviations apply: 

AAC-LC Advanced Audio Coding-Low Complexity audio object type 

ODG Objective Difference Grade 

PEAQ Perceptual Evaluation of Audio Quality 

SBR Spectral Band Replication 



General 



Conformance testing is an important tool to verify that implementations of Enhanced aacPlus match the relevant 
specifications. It is also helpful in verifying the proper use of the source code provided in 3GPP TS 26.410 [5] and 
3GPP TS 26.41 1 [6]. Conformance testing focuses on the core algorithm, therefore no criteria are defined for error 
concealment, downsampling and file I/O. 

Clause 5 describes the proposed method of conformance testing for the decoder. Clause 6 discusses encoder 
conformance. 



Decoder conformance testing 



Decoder conformance for both fixed-point and floating-point implementations shall be tested in accordance with 
clauses 5.1 and 5.2. Conformance shall be verified by either bit-exact behaviour to the reference output or by meeting 
the objective criteria defined below. Bit-exact behaviour should be preferred for fixed-point implementations where it 
can be achieved without undue penalty on computational complexity. 

5.1 AAC-LC and SBR conformance testing 

Conformance testing for AAC-LC and SBR shall be performed according to the relevant clauses in 

ISO/lEC 14496-4 [7] and ISO/lEC 14496-4 [8]. 3GPP conformance shall not be claimed for implementations achieving 

an accuracy level of K<15 for the 'RMS/LSB Measurement' test carried out for AAC-LC. 

5.2 Parametric Stereo conformance testing 

The 3GPP floating-point reference code implementation shall be used as the reference implementation. Conformance of 
an implementation shall be tested via objective quality assessments, using ITU-R Recommenadtion BS. 1387-1 [9]. An 
Objective Difference Grade comparison between the output of the reference decoder and the output of the decoder 
implementation under test shall result in a maximum ODG deviation of 0,25 for each single test vector.. The tests shall 
be run over bitrate configurations of 16 kbit/s, 21 kbit/s and 28 kbit/s, stereo. 

The set of test vectors shall be the same as used for characterization testing. The attached zip file 
"ps_bitstream_testvec.zip" contains the bitstream test vectors, a filename list given in Annex A. The attached zip file 
"ps_testvec_creation.zip" shall be used to derive the output test vectors. The zip file contains: 

• A Win32 precompiled version of the 3GPP floating-point decoder. 
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• A batch file which calls the reference decoder to produce reference decoded output test vectors for each 
bitstream test vector. 



6 Encoder conformance 

6.1 Floating point encoder 

No specific routines for floating-point encoder conformance testing are defined. It is recommended to use the 
floating-point code from 3GPP TS 26.410 [5]. In addition, it is recommended to verify that the implementation meets 
the criteria defined in clause 6.2 (Fixed-point encoder). If the floating-point code is used for an implementation in 
mobile equipment, the criteria defined in 6.2 shall be met. 



6.2 Fixed point encoder 



Conformance of fixed-point encoder implementations for use in mobile equipment shall be verified by bit-exact 
behaviour to the fixed-point reference code [6] as defined in 6.2.1, or by meeting the objective criteria as defined in 
6.2.2, or by performing subjective tests as described in 6.2.3. Fixed-point encoder implementations which are not used 
in mobile equipment, should meet the conformance criteria defined in this clause. 

Bit-exact behaviour should be preferred for fixed-point implementations where it can be achieved without undue 
penalty on computational complexity. 

If an implementer chooses to implement only a mono-encoder functionality, then conformance of only this functionality 
shall be tested. This shall apply irrespective of the conformance testing method chosen. 

6.2.1 Bit-exact Behaviour 

If the bit-exact conformance method is chosen, the conformance of a fixed-point encoder implementation under test 
shall be verified by checking the bit-exactness of the raw payload of the encoded output as produced by the fixed-point 
reference encoder code [6] for all 23 test vectors as given in Annex B. 



6.2.2 Objective criteria 



If the objective criteria conformance method is chosen, the conformance of a fixed-point encoder implementation under 
test shall be verified by checking objective conformance criteria for each of the three tools of the Enhanced aacPlus 
encoder: AAC, SBR and PS. The tests shall be conducted with the test vectors as given in Annex B. The configurations 
under test shall be 16 kbit/s mono, 32 kbit/s parametric stereo and 48 kbit/s stereo. All tests shall use the fixed-point 
encoder as reference. 

For the purpose of fixed-point encoder conformance testing a tool has been developed which can be used to extract the 
various parts of an Enhanced aacPlus bitstream such that conformance of each of those parts can be tested 
independently. The source code for this tool can be found in an electronic attachment (fixp_enc_test_tool.zip) to this 
document. For a description on how to invoke the various conformance tests, please refer to the 'readme.txt' file that is 
part of the attached package. This tool is henceforth referenced as Tixed-point encoder test tool'. 

6.2.2.1 AAC encoder conformance 

Conformance testing of the AAC encoder tool shall be conducted using an implementation of ITU-R BS.1387 (PEAQ) 
[9]. The test consists of a statistical analysis comparison of the reference encoder performance with the performance of 
the encoder under test. The following pseudo-code and the succeeding description outlines how the performance shall 
be assessed. 



cfg = 0; 

foreach (bitrate, channels, aacbandwidth) { 
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tv = 0; 
foreach (item) { 

bandlimit (item, item_bandlimited, aacband width); 

encoder_reference (item, reference_bitstream, bitrate, channels); 

encoder_test (item, test_bitstream, bitrate, channels); 

decoder_bandlimiting (reference_bitstream, reference_waveform); 

decoder_bandlimiting (test_bitstream, test_waveform); 

odg_ difference [cfg] [tv] = PEAQ_advanced (item_bandlimited, reference_waveform) - 

PEAQ_advanced (item_bandlimited, test_waveform); 

tv++; 

} 

cfg++; 



• 



• 



where: 

• bandlimit (in, out, bw) is a function that limits the bandwidth of the waveform 'in' to 'bw' and stores the result 
in the waveform 'out' 

• encoder_reference (in, out, br, ch) is a function that calls the reference encoder such that the waveform 'in' is 
encoded to the bitstream 'out' at the bitrate 'br' and using channelmode 'ch' 

encoder_test (in, out, br, ch) is a function that calls the encoder under test such that the waveform 'in' is 
encoded to the bitstream 'out' at the bitrate 'br' and using channelmode 'ch' 

decoder_bandlimiting (in, out) is a function that calls the fixed-point encoder test tool such that it decodes the 
AAC part of the bitstream "in" to the waveform "out", while limiting the bandwidth to the AAC part and 
performing QMF-domain upsampling 

• PEAQ_advanced (ref, deg) is a function that compares the reference waveform 'ref with the degraded 
waveform 'deg' returning the ODG resulting from that comparison 

The mean of the resulting vector odg_difference[cfg] shall not indicate an average performance of the encoder under 
test that is worse by more than 0.2. Moreover the worstcase difference shall not exceed 0.5. These criteria shall be 
fulfilled for all configurations. 

6.2.2.2 SBR encoder conformance 

Conformance for the SBR part of the encoder shall be measured for the various elements of the SBR bitstream 
independently. To eliminate the influence of potentially differing delays between the encoder under test and the 
reference encoder, the input waveforms shall be zero-padded at the beginning such that both encoders produce 
equivalent framing. 

The fixed-point encoder test tool shall be used to first extract the various SBR parameters that are embedded in the SBR 
bitstream under test and subsequently compare them to the output of the reference encoder. 

The following is an overview of how parameter extraction and parameter comparison work for the various modules. For 
further details, please consult the attached source code of the fixed-point encoder test tool: 

• The envelope energies signaled by the encoder under test are combined to the equivalent envelope energies for 
one envelope. This is done after dequantization and M/S -> L/R unmapping. The comparison however takes 
place in the log-domain. For higher energies only low deviations are allowed, while lower energies are 
permitted to deviate more: Differences of up to 3dB are allowed for all energies, differences of up to 6dB are 
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allowed for energies up to 42 dB below a full scale sine, differences of up to 9 dB are allowed for energies up 
to 60 dB below a full scale sine. 

• The test of the transient detector is performed by comparing the bitstream element sbr_grid() from both 
encoders. The RMS of the difference between the transient position vector of the encoder under test and the 
reference encoder shall not be greater than 0.2. 

• The test of the missing harmonics detector is carried out by comparing the bitstream elements 
sbr_sinusoidal_coding() from both encoders. The vector that is constructed by subtracting the missing 
harmonics vectors from both bitstreams and concatenating the result over all frames shall not have an RMS 
greater than 0.2. 

• The test of the whitening level detector is performed by comparing the bitstream element sbr_invf() from both 
encoders. The vector that is constructed by subtracting the whitening level vectors of the encoder under test 
and the reference encoder shall not have an RMS greater than 0.2. 

• The test for the noise floor estimator module is identical to the test for the envelope energy conformance test, 
except that the actual noise floor energies are compared instead of the envelope energies. The maximum 
permissible noise floor energy difference for a given deviation between reference encoder and encoder under 
test are as follows: Differences of up to 6 dB are allowed for all energies, differences of up to 9 dB are allowed 
for energies up to 42 dB below a full scale sine, differences of up to 12 dB are allowed for energies up to 60 
dB below a full scale sine. 

6.2.2.3 PS encoder conformance 

All comparisons shall be limited to frames and bands in the bitstream that contain an energy greater than -87 dB below 
a full-scale sine. Prior to the actual conformance test, the stereo parameters from both encoders are mapped to the 
highest available stereo frequency resolution. 

• the signal that is constructed from the differences between all IID values signaled by the encoder under test 
and the reference encoder shall have a maximum absolute difference of 2 and the RMS must not exceed 0.25 

• the signal that is constructed from the differences between all ICC values signaled by the encoder under test 
and the reference encoder shall have a maximum absolute difference of 2 and the RMS must not exceed 0.25 

6.2.3 Subjective criteria 

Subjective tests shall cover the encoder configurations tested during the characterization phase. The requirement for 
passing the subjective tests is that the encoder under test does not perform worse in a statistically significant sense in 
any test case when compared to the fixed-point reference encoder. 
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Annex A (normative): 

Bitstream test vectors for testing Parametric Stereo 

decoding 

Table 1 : Test vectors for Parametric Stereo decoding 



# 


Bitstream test vector file name 


Bitrate configuration 


1 


m cl X 2 16s.3gp 


1 6 kbit/s 


stereo 


2 


m ot X 1 16s.3gp 


1 6 kbit/s 


stereo 


3 


m p X 1 16s.3gp 


1 6 kbit/s 


stereo 


4 


m si X 1 16s.3gp 


16 kbit/s 


stereo 


5 


s cl 2t 1 16s.3gp 


1 6 kbit/s 


stereo 


6 


s cl 2t 2 16s.3gp 


16 kbit/s 


stereo 


7 


s cl mt 1 16s.3gp 


1 6 kbit/s 


stereo 


8 


s no ft 2 16s.3gp 


16 kbit/s 


stereo 


9 


sbm sm x 1 16s.3gp 


1 6 kbit/s 


stereo 


10 


sbm sm X 2 16s.3gp 


1 6 kbit/s 


stereo 


11 


som fi X 2 16s.3gp 


16 kbit/s 


stereo 


12 


som ot X 1 16s.3gp 


1 6 kbit/s 


stereo 


13 


m cl X 2 21s.3gp 


21 kbit/s 


stereo 


14 


m ot X 1 21s.3gp 


21 kbit/s 


stereo 


15 


m p X 1 21s. 3gp 


21 kbit/s 


stereo 


16 


m si X 1 21s.3gp 


21 kbit/s 


stereo 


17 


s cl 2t 1 21s.3gp 


21 kbit/s 


stereo 


18 


s cl 2t 2 21s.3gp 


21 kbit/s 


stereo 


19 


s cl mt 1 21s.3gp 


21 kbit/s 


stereo 


20 


s no ft 2 21s.3gp 


21 kbit/s 


stereo 


21 


sbm sm x 1 21s.3gp 


21 kbit/s 


stereo 


22 


sbm sm X 2 21s.3gp 


21 kbit/s 


stereo 


23 


som fi X 2 21s.3gp 


21 kbit/s 


stereo 


24 


som ot X 1 21s. 3gp 


21 kbit/s 


stereo 


25 


m cl X 2 28s.3gp 


28 kbit/s 


stereo 


26 


m ot X 1 28s.3gp 


28 kbit/s 


stereo 


27 


m p X 1 28s. 3gp 


28 kbit/s 


stereo 


28 


m si X 1 28s.3gp 


28 kbit/s 


stereo 


29 


s cl 2t 1 28s.3gp 


28 kbit/s 


stereo 


30 


s cl 2t 2 28s.3gp 


28 kbit/s 


stereo 


31 


s cl mt 1 28s.3gp 


28 kbit/s 


stereo 


32 


s no ft 2 28s.3gp 


28 kbit/s 


stereo 


33 


sbm sm x 1 28s.3gp 


28 kbit/s 


stereo 


34 


sbm sm X 2 28s.3gp 


28 kbit/s 


stereo 


35 


som fi X 2 28s.3gp 


28 kbit/s 


stereo 


36 


som ot X 1 28s. 3gp 


28 kbit/s 


stereo 
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Annex B (normative): 

Test vectors for fixed-point encoder conformance 

Table 2: Testvectors specifically relevant for the objective testing of the AAC tool 



# 


Test vector file name 


1 


ct castagnettes.wav 


2 


ct glockenspiel.wav 


3 


ct test 1 48.wav 


4 


ct test 2 48.wav 


5 


ct test 3 48.wav 


6 


cymbal.wav 


7 


gong.wav 


8 


guitar l.wav 


9 


guitar 2.wav 


10 


guitar cymbals.wav 


11 


handcuff.wav 


12 


hihat.wav 


13 


laugli.wav 


14 


shal<er.wav 


15 


speecli female.wav 


16 


speecli male qerman.wav 


17 


speecli male portuguese.wav 


18 


triangle.wav 


19 


wooden toys l.wav 


20 


wooden_toys_2.wav 



Table 3: Testvectors specifically relevant for the objective testing of the SBR tool 



# 


Test vector file name 


SBR module 


21 


noise fade.wav 


Envelope 


12 


hihat.wav 


Transients 


1 


ct castagnettes.wav 


Transients 


22 


sines.wav 


Sines 


21 


noise fade.wav 


Noise 


23 


invf.wav 


Whitening 



Table 4: Testvectors specifically relevant for the objective testing of the PS tool 



# 


Test vector file name 


PS parameter 


3 


ct test 1 48.wav 


IID 


2 


ct glockenspiel.wav 


IID 


3 


ct test 1 48.wav 


ICC 


18 


triangle.wav 


ICC 
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Annex C (informative): 
Change history 



Change history 


Date 


TSG SA# 


TSG Doc. 


CR 


Rev 


Subject/Comment 


Old 


New 


2005-06 


28 


SP-050242 






Approved at TSG SA#28 


2.0.0 


6.0.0 


2005-12 


30 


SP-050786 


0001 


2 


Correction to Enhanced aacPlus Encoder conformance 


6.0.0 


6.1.0 


2006-03 


31 


SP-060013 


0002 




Correction to Enlianced aacPlus Decoder conformance: 
replacement of one test-vector 


6.1.0 


6.2.0 


2006-06 


32 


SP-060354 


0004 


1 


Correction to Enlianced aacPlus Decoder conformance: 
clarification of conformance criteria 


6.2.0 


6.3.0 
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